Simulations of Ground State Fluctuations in Mean-Field Ising Spin Glasses 
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The scaling of fluctuations in the distribution of ground-state energies or costs with the system 
size A'^ for Ising spin glasses is considered using an extensive set of simulations with the Extremal 
Optimization heuristic across a range of different models on sparse and dense graphs. These models 
exhibit very diverse behaviors, and an asymptotic extrapolation is often complicated by higher- 
order corrections. The clearest picture, in fact, emerges from the study of graph-bipartitioning, a 
combinatorial optimization problem closely related to spin glasses. Aside from two-spin interactions 
with discrete bonds, we also consider problems with Gaussian bonds and three-spin interactions, 
which behave differently to a significant degree. 



I. INTRODUCTION 

Ising spin glasses are the paradigmatic model for dis- 
order not only in materials [ij, but provide the archetype 
for complex behavior in many contexts, such as hard com- 
binatorial problems 0, information theory and 
learning f3,% An Ising spin glass is generally described 
by the Hamiltonian 



H 



(1) 



with N Ising spins (Ji G {±1} and quenched random 
bonds Ji The bonds are chosen from a distribution, 



here, of zero mean and width -y/ ( J^) = Jq that is either 
bimodal {Jij S {zLJq}) or Gaussian. The sum parses 
over all extant bonds < i,j > between any pair of spins 
ai and aj. The Sherrington-Kirkpatrick model (SK) Q 
is the mean-field limit of Ising spin glasses, in which each 
spin is coupled to every other spin in the system. To keep 
the energies E of an equilibrium spin configuration in SK 
extensive, we set Jq = 1/ ; the other models discussed 
here are defined on sparse graphs and require Jq — 1. 

It is surprising to find still new features of such a well- 
studied model after some 30 years study. Therefore, the 
unusual behavior of the distribution of ground-state en- 
ergies Eq over the bond disorder in SK has raised sig- 
nificant interest in recent years. This interest is further 
elevated by its close connection with the statistics of ex- 
tremely rare events, that in many jammed, disordered 
systems can become the controlling feature of the dy- 
namics. It was found (8l-[T2| that the fluctuations in E^ 
behave in a highly non-normal fashion and rather resem- 
ble distributions found in extremal- value statistics [l3| . 
This connection becomes rigorous for the fluctuations of 
the random-energy model (REM) [13], for which a Gum- 
bel distribution can be derived exactly. The shape in 
SK shares similarities with a higher-order Gumbel dis- 
tribution [1, [ill, [IB| but its precise functional form re- 
mains unknown. There exists a high degree of univer- 



sality in the extreme-value statistics of intrinsically un- 
correlated states, but in long-range connected systems 
with quenched disorder such an assumption may fail. A 
general discussion of the ordinary versus extreme fluctu- 
ations in the SK and other disordered models is provided 
in Ref. [16]. 

Of special interest for the energy fluctuations is the 
variation in width of the distribution with the system 
size, as it provides important clues to the structural prop- 
erties of the ground states (or low temperature states 
generally) IitI . ITsj . A number of arguments regarding 
the system-size scaling of the standard deviation for the 
ground-state energy densities eg — Eq/N, 



f7(eo) = \J {el) - {eof 



(2) 



(a > p) , 
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for — >■ OO have been put forward, leading to values of 
either 0, [H p = f or P = §• Both conjectures 

predict decay that is faster than for normal fluctuations 
p = i, which would be obtained from the central limit 
theorem under the assumption of negligible correlations 
between individual terms of the spin glass Hamiltonian in 
Eq. (H]). While a bound of p > | has been shown [TtIITsI 
for the SK, Gaussian behavior is indeed found for spin 
glasses on a flnite-dimensional lattice |26l|. and has also 
been observed on sparse random graphs [9|. Exact val- 
ues different from any of these are known for the replica 
symmetric spherical spin glass (p = |) and the m- vector 
spin glass for m — > oo (p = |) |27fl . 

Numerically, both conjectures for p on the SK have 
proven difficult to distinguish with any certainty, and 
while most initial predictions (8l-[ll| seems to favor a 
value close to p = | , more recently a trend towards p = | 
was found at larger system sizes fl^l. That would sup- 
port the current consensus in the theoretical work ^211 - 
l23fl . Such a larger value is also desirable for consistency 
between relations connecting p to the exponent describ- 
ing domain wall excitations ^19] as found in high dimen- 
sions mi. 

Here, we report on extensive simulations to clarify this 
important question regarding the low-temperature prop- 
erties of spin glasses. The results are at best marginally 
consistent with any of the theoretical predictions. Since 
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the data analysis proves to be complicated by transient 
behavior, we have widened the scope of our investigation 
to incorporate a large number of related models for com- 
parison. For instance, we provide corresponding data 
for spin glasses on random regular graphs ("Bethe lat- 
tices") of sparse degree, both for two- and three-spin in- 
teractions and discrete (± J) as well as Gaussian bonds. 
Two-spin coupled spin glasses on Bethe lattices of degree 
r provide a convenient one-parameter family of models 
with smoothly extrapolates to SK for increasing degree, 
r — > oo [29, 30]. The results are quite similar to those we 
find in SK, thus, providing a likely trend for the SK be- 
havior itself, but they are equally beset with strong tran- 
sients. The three-spin data with discrete bonds draws 
a different picture, consistent with a recent study [sT'], 
and also highlights the question of universality of the re- 
sults when continuous bonds are used. As we have noted 
before [3^ . on sparse graphs finite-size corrections may 
already depend on details of the bond distribution; this 
dependence appears to extend also to ground-state devi- 
ations. 

Alternatively, we study the graph bipartitioning prob- 
lem (GBP) on these Bethe lattices. We find much dimin- 
ished transients and a consistent extrapolation for the 
value of p. But that value is between - and likely distinct 
- from I and |. Although it was recently predicted [33] 
that GBP in the thermodynamic limit is equivalent to 
the corresponding spin glass at T — on those Bethe 
lattices, it is of course less clear whether such relation 
would hold at finite size. For instance, finite-size correc- 
tions to the average ground state energies already differ 
significantly between GBP and spin glasses. 

This paper is structured as follows: We start with a 
few remarks about the optimization heuristic used in all 
of our simulations in Sec. Ill Al Since the clearest case 
is provided by GBP, we present our data for this prob- 
lem first in Sec. IIIII In Sec. IIVI we discuss the SK data 
at length. In Sec. |Vl we present the data for the corre- 
sponding spin glass problem on Bethe lattices, followed 
by a similar study on ordinary random graphs in Sec. lVIl 
In Sec. lVIII we supplement our investigation with a study 
of three-spin interactions on a Bethe lattice. We summa- 
rize with a few conclusions in Sec. I Villi 



II. MEANS AND METHODS 

In this section, we introduce the simulation methods 
and techniques by which the resulting data was analyzed. 

A. Optimization Methods 

We have employed the Extremal Optimization heuris- 
tic (EO) |34l436i] in the implementation described previ- 
ously for the SK spin glass [11] and those on Bethe lat- 
tices [30| , with various improvements to attain an order 
of magnitude in speed-up [60] . Resorting to a simple bi- 



modal bond distribution whenever possible allows further 
an efficient use of integer arithmetic. Not only do discrete 
bonds provide computational advantages over continuous 
ones; Gaussian bonds pose significant entropic barriers in 
addition to the usual complexities faced by local search 
in a multi- modal energy landscape (37| . To find approx- 
imations to GBP ground states, we use EO as described 

in Ref. |3l,[3- 

Unlike, for instance, the parallel tempering Monte 
Carlo technique used at larger system sizes for SK in 
Ref. [l^ . which operates at small but finite temperatures, 
EO generally performs its local search for minima in the 
landscape formed by the internal energy itself, with acti- 
vated spin flips as elementary moves [S^- The only free 
parameter controlling EO was set to r = 1.25 for discrete 
problems and somewhat higher at r = 1.5 for problems 
with Gaussian weights, and about 0.1 spin flips were 
executed in each run, restarting form a random initial 
conflguration. At least 2 such restarts were performed 
for each instance, and the number of runs is doubled on- 
the-fly whenever a new optimum is found in the latter 
half of all runs. For some rare instances, more than 10 
runs were required, always ensuring that the latter half 
of all runs merely confirms the previously found optimum 
but does not exceed it. In each section, we have listed the 
number of instances treated at each N . Besides trying to 
reach large sizes, we have often emphasized simulating a 
very large number of instances on a relatively dense set of 
intermediate iV. We have conducted extensive tests for 
each model to ensure the accuracy of the results. Testing 
on testbeds of exactly-solved instance (using branch-and- 
bound) typically results in perfect agreement, but system 
sizes are small {N « 50 — 70). On larger systems, we have 
done sample runs with ten-times more updates and found 
only few inaccuracies, which lead to systematic errors far 
below statistical errors (unless otherwise noted). 

Although all of the problems in this project could be 
classified as spin glasses, they stand in for a large class 
of combinatorial problems. It should be noted that the 
EO heuristic provides data of great detail with only small 
changes in the implementation. Only the input for the 
various graph types needed to be changed. (For GBP, 
we have to impose the additional constraint of vanishing 
magnetization.) While we treat only mean- field prob- 
lems here, pertinent results have been obtained previ- 
ously for structured instances, such as finite-dimensional 
lattices [solliot. 



B. Data Analysis 

To obtain insights into the finite-size scaling of our ob- 
servables, we study fits of the data generated in our sim- 
ulations to a number of asymptotic forms. These forms 
seem to be the most nature candidates to describe cor- 
rections to the leading asymptotic behavior. But there 
can never be an exhaustive list of all possibilities without 
any theoretical knowledge a-priori. Therefore, we list as 
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much as possible the data points obtained in the simu- 
lations in tables that would allow the readers to pursue 
their own hypotheses. 

Any asymptotic {N — > oo) fit bares considerable risks: 
Not only may a presumed asymptotic form be insuffi- 
cient and may, for instance, miss logarithmic corrections, 
etc. But even if correct, it would certainly fit data for 
larger system sizes N better than any transients, rais- 
ing the question of how many data points for lower N to 
include. Any of these uncertainties can introduce poten- 
tially sizable systematic errors, even if the heuristic had 
been perfectly accurate. Such errors can only be elimi- 
nated fully with a theory to compare to, which at present 
does not exist. The obtained qualities-of-fit Q have to be 
seen in this context, and a poor Q-value should not be 
taken immediately as disproving a hypothesis, unless it is 
seen in relation to alternative fits and a discussion of how 
much transient data has been included. In fact, to obtain 
any reasonable Q- values, in each fit we have - somewhat 
arbitrarily - increased the error bars uniformly by a fac- 
tor of four from the purely statistical errors given as a 
bracketed uncertainty in the last digit of the simulation 
data listed in the tables. Such a uniform allowance for 
systematic errors, either from heuristic inaccuracies or 
functional uncertainty in the fit, is definitely inadequate 
and would deserve better consideration in the future. 

Thus, we fit the asymptotic extrapolation of the finite- 
size data towards the thermodynamic limit {N — > oo) for 
the presumed ground-state energy (e, ) „ or cost densities 



N 



to the following forms, abbreviating x G {c, e}: 



AN- 



for just the first-order correction, or 



AN- 



(3) 



(4) 



as a second order correction, where we have to fix the 
first-order exponent uj and require wi > w to achieve a 
stable fit. An alternative second-order, logarithmic fit. 



AN- 



B- 



IniV 

IT 



(5) 



with arbitrary uj < 1 also seems conceivable. 

The asymptotic scaling for the respective deviation ex- 
ponent p proceeds either through a direct fit (to first- or 
second order) to Eq. ([2]), or by the corresponding extrap- 
olated form 



logiV 



logC 
logiV 



O 



logiV 



(6) 



Either method has its advantages and disadvantages. A 
direct fit, plotted on a double-logarithmic scale, provides 
easier convergence when higher-order terms are included. 
But the logarithmic scale smoothes out variability or 
transient in a data set. In turn, the extrapolated form 
in Eq. ([5]) is far less forgiving and provides significantly 
more insight into fluctuations in the data set. When plot- 
ted on an 1/ logiV-scale, data that extends over decades 



is squashed into a relatively small space. But if a solid 
linear regime emerges, it provides stronger evidence for 
the existence and value of an exponent than a double- 
logarithmic plot. The slope of this extrapolation is ar- 
bitrary, though, as we could multiply a by any constant 
factor without affecting the scaling. We may choose to 
fix this factor in a way (such as for the spin glasses be- 
low) that the log C-term in Eq. ^ vanishes, or (such as 
for the GBP below) to splay out several plots into one 
graph. Clearly, such a choice does not alter any linear 
regime, if it exists. Without such a linear regime, as we 
will see, it is very difficult to extract satisfactory informa- 
tion from such a fit, even when higher-order (and strongly 
non-polynomial) corrections are taken into account. 

As we have argued in Ref. [33|, under certain circum- 
stances it is necessary to consider the cost of frustration 
(sum of all violated bonds) in a ground state, instead of 
its energy (difference between all violated and all unvio- 
lated bonds). We will switch in our discussion between 
cost and energy repeatedly. There is a simple linear re- 
lation between their average densities. 



(eo)^ = 2(co)^ - - (|J|)^, 



(7) 



where r is the (average or fixed) degree and p refers to 
the number of spins coupled via a single bond. The aver- 
age absolute bond- weight for the Gaussian distribution of 
zero mean and standard width ( Jq = 1 on sparse graphs. 



Jq — 1/N for SK) is given by limjv_ 
whereas it is simply (|J|) 



N 



N 



1 for a bimodal distri- 
bution on a sparse graph. While there is no difference 
between cost and energy at the level of averages, the 
corresponding relation of the (co-)variances mixes the 
fluctuations in cost and energy with that of the abso- 
lute bond weights. If in each instance the absolute sum 
of all bond weights is the same, i. e. a (5-peak with- 
out fluctuations, cost and energy fluctuations are pro- 
portional. But if the absolute bond weight sum fluc- 
tuates (independently) from instance to instance, either 
energy- or cost-fluctuations will become equally normal- 
distributed, which leaves at most one of them non-trivial. 
Typically, for SK cost-fluctuations are trivial (i. e. nor- 
mal) and energy-fluctuations are interesting, while for 
sparse graphs with variable degree and/or continuous 
bonds the role reverses. Only for Bethe lattices (of fixed 
degree) with bimodal bonds, cost and energy-fluctuations 
are proportional. When r 3> 1 and each spins acquires a 
nearly extensive number of neighbors, matters get more 
complicated. 



III. GRAPH BI-PARTITIONING ON BETHE 
LATTICES 

In GBP, a graph of (even) N vertices and a certain 
number of edges is divided into two sets of equal size N/2 
such that the number of edges connecting both sets, the 
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Table I: Average cost per spin (cr) for approximate ground states of GBP on Bethe lattices of degree r = 3, . . . , 10. The 
given errors, in parentheses after each average, denote the uncertainty in the last given digit of that average. This uncertainty 
is solely based on the statistical error. Fits of this data and their plots were discussed previously in Ref. ^331] (see Fig. 2 there). 
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(07) ,T 

\ ' ' N 




(cq) ,t 


\»-'iu/ 


32 


19495f3l 


35652('4^ 


53893f5l 


71445('5) 


91552f6"l 


1 09781(6) 


1 31056f6l 


1 49700('7) 


40 


0.18161(3) 


0.34104(3) 


0.51730(4) 


0.69348(4) 


0.88800(5) 


1.07190(5) 


1.27774(5) 


1.46657(6) 


50 


0.16627(2) 


0.32820(3) 


0.49487(3) 


0.67581(4) 


0.85919(4) 


1.05003(4) 


1.24320(5) 


1.44071(5) 


64 


0.15822(2) 


0.31632(2) 


0.48312(3) 


0.65972(3) 


0.84438(3) 


1.02997(4) 


1.22578(4) 


1.41715(4) 


80 


0.15042(1) 


0.30750(2) 


0.47119(2) 


0.64753(3) 


0.82899(3) 


1.01495(3) 


1.20743(3) 


1.39936(3) 


100 


14406f2'l 


30016('2") 


46138f3'l 


63749('3~) 


81632f3'l 


1.00232(4) 


1.19222(4) 


1 38449(4") 


128 


0.13825(2) 


0.29350(1) 


0.45240(2) 


0.62822(2) 


0.80481(3) 


0.99082(3) 


1.17837(3) 


1.37085(3) 


160 


0.13411(1) 


0.28858(2) 


0.44582(2) 


0.62133(2) 


0.79628(2) 


0.98220(2) 


1.16812(3) 


1.36075(3) 


200 


0.13064(1) 


0.28450(1) 


0.44037(2) 


0.61553(2) 


0.78920(2) 


0.97498(2) 


1.15965(2) 


1.35227(3) 


256 


0.12753(1) 


0.28087(1) 


0.43545(1) 


0.61037(2) 


0.78281(2) 


0.96852(2) 


1.15195(2) 


1.34449(2) 


320 


0.12526(1) 


0.27818(1) 


0.43185(2) 


0.60652(2) 


0.77813(2) 


0.96370(2) 


1.14630(2) 


1.33874(3) 


512 


0.12170(1) 


0.27395(1) 


0.42607(1) 


0.60054(2) 


0.77073(2) 


0.95605(2) 


1.13734(2) 


1.32968(2) 


1024 


0.11852(1) 


0.26993(1) 


0.42075(1) 


0.59497(2) 


0.76407(4) 


0.94948(4) 


1.12965(5) 


1.32211(6) 


2048 


0.11670(4) 

















"cutsize", is minimized. The global constraint of an equal 
division places the GBP among the hardest problems in 
combinatorial optimization, since determining the exact 
solution with certainty would in general require a conipu- 
tational effort growing faster than any power of A'^ |41| . 
Applications of graph partitioning reach from the design 
of integrated circuits (VLSI) [4j| to the partitioning of 
sparse matrices (43| , leading to very different requirement 
regarding the mix of speed and accuracy in a heuris- 
tic [44]. In Refs. [2^, |4^, we have considered a range 
of different graph ensembles, which can affect character- 
istics of GBP drastically; here, as in Ref. [s^l, we merely 
focus on Bethe lattices, which are locally tree-like such 
that some analytical results have been derived [47-53|- 
These Bethe lattices are also known as r-regular graphs, 
and are generated by fixing the degree r at each vertex, 
which in turn is connected to other vertices at random. 



A. Average Ground State Costs 

In Tab.lll we have obtained approximate optima in the 
cutsize per vertex (cr-)^ on Bethe lattices for degrees r 
between 3 and 10, and graph sizes between iV = 32 and 
2048. Statistical errors of our averages have been kept 
small by generating a large number of instances for each 
N and r, typically « 10^ for N < 200, nj « 10^ for 

> 256. Ref. ^] described the implementation of the 
r-EO heuristic used to obtain the ground state approx- 
imations for GBP. We have presented some of the most 
salient results of the extrapolation to the thermodynamic 
limit there. 

In Ref. [13, we used a fit with first-order corrections 
according to Eq. ^ alone, see Tab. 1 and Fig. 2 there. 
Such a first-order fit to the data from the largest system 
sizes only is discussed in Tab. |TT1 While the fits produce 



rather consistent results, with very similar values for lu, 
there is clearly a distinction between even and odd r in 
the quality of the fits, indicative of parity effects typical 
of regular graphs already noted earlier [29l . [sj ■ 

Table II: Fit of the data in Tab. |I] to Eq. © for the cost c. 
Only data for 64 < < 2048 has been utilized. 



r 




A 


LU 


ndf 


xVndf 


Q 


3 


0.114765 


1.71 


0.88 


6 


0.178 


0.98 


4 


0.265240 


1.77 


0.85 


6 


2.124 0.047 


5 


0.414254 


2.35 


0.85 


6 


0.410 


0.87 


6 


0.587974 


2.37 


0.84 


6 


0.622 


0.71 


7 


0.755752 


3.11 


0.86 


6 


0.138 


0.99 


8 


0.940334 


2.93 


0.84 


6 


1.053 


0.39 


9 


1.119390 


3.74 


0.86 


6 


0.599 


0.73 


10 


1.310780 


3.37 


0.83 


6 


1.836 


0.088 



Table III: Fit of the data in Tab.Uto Eq. (0) with fixed a; = |. 
Only data for 64 < TV < 2048 has been utilized. 



r 




A 


B 


ndf xVndf 


Q 


3 


0.116574 


-0.69 


-1.29 


6 


6.347 


l.le-06 


4 


0.266936 


-0.56 


-1.28 


6 


11.886 


2.2e-13 


5 


0.416543 


-0.75 


-1.73 


6 


6.770 


3.4e-07 


6 


0.590277 


-0.71 


-1.75 


6 


2.311 


0.031 


7 


0.759679 


-1.21 


-2.45 


6 


2.585 


0.017 


8 


0.944030 


-1.02 


-2.29 


6 


0.645 


0.69 


9 


1.124500 


-1.52 


-3.01 


6 


1.747 


0.11 


10 


1.315190 


-1.14 


-2.66 


6 


0.600 


0.73 



A surprising feature of these fits are the values for 
the finite-size scaling exponent uj. In our previous stud- 
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Figure 1: Plot of the PDFs for the ensemble fluctuations of the optimal costs per spin for GBP on Bethe lattices of degree 
r = 3, 4, . . . , 10. When rescaled by their deviation a = a {cr)j^, the data collapses virtually for all system sizes onto a highly 
skewed master-curve. There appears to be little change in skewness between degrees r. 
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Table IV: Fit of the data in Tab. [T] for all to Eq. @, but 
restricting uji — 2ijj. 
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ndf xVndf Q 
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A 1 1 /I ono 
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0.76 
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0.940714 


3.33 


-4.55 
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1.116430 
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137.050 
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1.311290 
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-4.83 


0.85 


9 
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0.46 



ies |29l . |30| of bimodal spin glasses on Bethe lattices we 
found throughout that this exponent was most consis- 
tent with a; = |, which has also been predicted for SK, 
see Sec. IIV Al While it was argued in Ref. [s^ that bi- 
modal spin glass and GBP on Bethe lattices should be 
equivalent in the ground state energy, this equivalence 
apparently does not extend even to first-order correc- 
tions, as the values for w in these fits are significantly 
higher. To analyze whether even higher-order corrections 
could rectify this discrepancy in scaling, we attempt a fit 
of the form in Eq. ^ with w = § fixed but variable B 
and uji. Such a fit fails to converge, as in each iteration 
uji moves closer to w. Alternatively, we prescribe the 
form of a plausible logarithmic higher-order correction 
in Eq. The resuhs of this fit are listed in Tab. [ml 
The quality of the fit is very poor for small degree r 
but gets progressively better for increasing r, suggesting 
a possible approach to the SK result for w for r — ^ cx). 
Yet, the results for (cr)^ are somewhat less in agreement 
with the conjectured equivalence in Ref. [33]. Generally, 
if the next-order correction contains a (possibly polyno- 
mial) logarithmic dependence of this sort, any attempt 
to predict lo may be futile. 

Another alternative form to fit is provided by a series 
expansion in powers of iV~'^. Thus, fitting to Eq. ^ with 
uji = 2cj we obtain the results listed in Tab. IIVI Now, 
the results for (cr)^^ are again in good agreement with 
the conjecture but the quality of the fit is not improved 
over the mere first-order fit in Tab. [TTl Interestingly, this 
form is now a better fit with even values for r but terrible 
for odd ones, reversing the trend from the previous fits. 
Correction terms may differ between even and odd r not 
only in their constants but in their very form. 



B. Ground State Fluctuations 

In Fig. [Tl we plot the probability density functions 
(PDF) of ground-state costs over the ensembles of Bethe 
lattices. Unlike for the spin glass problems below, tran- 
sients due to finite-size effects diminish quickly. This 
leads to a solid collapse of the data when properly 
rescaled by their deviations cr(cr)jv in Eq. appar- 
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Figure 2: Extrapolation in the limit N ^ oo for the expo- 
nent p, defined in Eq. ([SJ, from the deviations a{cr)j^ in 
Tab. |V] obtained from the distribution of GBP ground state. 
Appropriately rescaled via Eq. the data for each de- 
gree r extrapolates linearly to the thermodynamic limit, with 
rapidly diminishing transients. Note that for each r the fit- 
ted region spans at least one decade in system size N. The 
goodness-of-fit is Q = 1 for all r except for r = 3, where 
Q ^ 0.73. 




0.05 0.1 0.15 0.2 0.25 0.3 
1/r 

Figure 3: Extrapolated values of p for each degree r, obtained 
from Fig. [2l plotted as a function of inverse degree. For even 
and odd values of r separately, each sequence extrapolates to 
value of poo = 0.77(2) for r — >■ oo. The error bars result from 
the fits in Fig. [51 systematic errors, e. g. due to higher-order 
corrections ignored in those fits, are potentially larger. 



ently with little difference in the scaling function even 
for varying degrees r. 

In Tab. IVl we list the deviations a (cr)^, as defined in 
Eq. for each degree r and system size N. A closer 
look at these deviations indeed provides a robust extrap- 
olation for the exponent p. Eq. ([Blshould yield a linear ex- 
trapolation for the exponent p when plotted vs. 1 / log N, 
assuming negligible corrections (i. e. N~°' ^ N^p). In 
Fig. [21 we show these plots for all values of r. It is ap- 
parent that transient behavior decays very quickly, and a 
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Table V: Standard deviation of the ground-state costs in GBP. Statistical errors are estimated as (r/y^rTr, where nj refers to 
the number of instances considered. Appropriately rescaled, this data is plotted also in Fig. [5] (Some of the data, in particular, 
for larger A*' and r, shows an obvious systematic bias due to two likely sources: a bad approximation for some ground states 
and an undercount of rare instances with costs extremely far from the average. These data points have been left out of Fig. [2]) 



N 


cr(c3)]v 


cr(c4)jv 


0-(c5)]V 


cr(c6)jv 


cr(c7)]V 


o-{cs)n 


0-(c9)jv 


(j{cio)n 


32 


0.03206(3) 


0.04108(4) 


0.04561(5) 


0.05222(5) 


0.05598(6) 


0.06118(6) 


0.06470(6) 


0.06910(7) 


40 


0.02664(3) 


0.03428(3) 


0.03814(4) 


0.04370(4) 


0.04680(5) 


0.05129(5) 


0.05419(5) 


0.05790(6) 


50 


0.02252(2) 


0.02860(3) 


0.03215(3) 


0.03653(4) 


0.03953(4) 


0.04301(4) 


0.04564(5) 


0.04869(5) 


64 


0.01784(2) 


0.02333(2) 


0.02615(3) 


0.02998(3) 


0.03228(3) 


0.03539(4) 


0.03747(4) 


0.04009(4) 


80 


0.01482(1) 


0.01945(2) 


0.02192(2) 


0.02511(3) 


0.02710(3) 


0.02967(3) 


0.03146(3) 


0.03367(3) 


100 


0.01253(2) 


0.01622(2) 


0.01837(3) 


0.02096(3) 


0.02280(3) 


0.02490(4) 


0.02639(4) 


0.02825(4) 


128 


0.01012(2) 


0.01329(1) 


0.01517(2) 


0.01725(2) 


0.01877(3) 


0.02051(3) 


0.02179(3) 


0.02329(3) 


160 


0.00847(1) 


0.01113(2) 


0.01269(2) 


0.01445(2) 


0.01573(2) 


0.01716(2) 


0.01829(3) 


0.01948(3) 


200 


0.00712(1) 


0.00935(1) 


0.01066(2) 


0.01209(2) 


0.01323(2) 


0.01442(2) 


0.01536(2) 


0.01641(3) 


256 


0.00582(1) 


0.00769(1) 


0.00878(1) 


0.00996(2) 


0.01092(2) 


0.01183(2) 


0.01271(2) 


0.01351(2) 


320 


0.00490(1) 


0.00643(1) 


0.00734(2) 


0.00835(2) 


0.00919(2) 


0.00995(2) 


0.01069(2) 


0.01140(3) 


512 


0.00336(1) 


0.00441(1) 


0.00509(1) 


0.00580(2) 


0.00638(2) 


0.00697(2) 


0.00746(2) 


0.00798(2) 


1024 


0.00195(1) 


0.00253(1) 


0.00295(1) 


0.00339(2) 


0.00386(4) 


0.00428(4) 


0.00463(5) 


0.00506(6) 



smooth linear extrapolation is obtained. Also re-assuring 
is the fact that this data shows a small even/odd effect, 
just as for the costs above in Sec. IIII Al The sequence of 
extrapolants for p appear to converge towards the same 
value of p = 0.77(2), at least for r — >■ oo. It is surpris- 
ing to find such an apparently non-trivial result in such 
a simple mean-field model. Comparing with theory, it 
comes closest to the value of p = | previously proposed 
in Ref. [9] and seems far from that proposed for the SK 
model, p = I [2^ - [25| . There is. of course, no reason for 
it to be equal to that of SK. But the in-between value 
found here for GBP is arguably close to the correspond- 
ing extrapolations for the spin glasses below, which by 
themselves remain inconclusive due to strong transients 
up to large system sizes. 

IV. SHERRINGTON-KIRKPATRICK MODEL 

We reconsider finite-size corrections in the SK model 
by expanding on the simulations in Ref. [ll| with sub- 
stantially more instances at many more system sizes, see 
Tab. IVII It is found that the corrections to the average 
energies permit fits of reasonable quality with system- 
atic improvements when higher-order corrections are in- 
corporated, but the discussion of the energy fluctuations 
remains largely inconclusive. Interestingly, this scenario 
is the converse of that found in Sec. Illll for GBP. 



A. Average Ground State Energy 

First, we want to consider the average energies, which 
provide an assessment of the quality of our simulations in 
reference to exactly known results about SK; a necessary 
step preliminary to a more demanding analysis of the 



Table VI: List of all the data obtained with EO in sequence 
of system size A''. Given are the number of instances nj con- 
sidered at each A', and the average ground-state energy den- 
sity {eo)jv and average standard deviation a{eo) over these 
instances. 



N 


m 




o- (eo) 


N 


71/ 


(eo)Ar 


o- (eo) 


15 


5000000 


-0.64449(3) 


0.0670 


127 


732463 


-0.73538(2) 


0.0147 


17 


5000000 


-0.65441(3) 


0.0614 


149 


723526 


-0.73828(2) 


0.0130 


21 


5000000 


-0.66931(2) 


0.0531 


169 


624094 


-0.74029(2) 


0.0119 


25 


5000000 


-0.67994(2) 


0.0471 


199 


351317 


-0.74273(2) 


0.0104 


31 


1480000 


-0.69127(3) 


0.0407 


225 


329043 


-0.74424(2) 


0.0095 


35 


5000000 


-0.69701(2) 


0.0374 


255 


255572 


-0.74587(2) 


0.0086 


41 


5000000 


-0.70371(2) 


0.0335 


299 


454555 


-0.74759(1) 


0.0077 


49 


5500000 


-0.71049(1) 


0.0295 


349 


317264 


-0.74910(1) 


0.0068 


55 


5000000 


-0.71442(1) 


0.0272 


399 


204045 


-0.75030(1) 


0.0061 


63 


5000000 


-0.71872(1) 


0.0246 


511 


51246 


-0.75233(2) 


0.0050 


69 


1000000 


-0.72113(2) 


0.0232 


549 


144912 


-0.75278(1) 


0.0048 


79 


5000000 


-0.72505(1) 


0.0208 


649 


59717 


-0.75383(2) 


0.0042 


89 


1000000 


-0.72783(2) 


0.0192 


799 


25257 


-0.75491(2) 


0.0037 


99 


800000 


-0.73043(2) 


0.0177 


1023 


5338 


-0.75615(4) 


0.0031 


109 


816132 


-0.73242(2) 


0.0165 


2047 


403 


-0.7583(1) 


0.0020 



data. In Fig. [5l we plot the data for the average ground- 
state energy density (eo)^ from Tab. IVII for the different 
N. It has been argued on the basis of theoretical stud- 
ies (s^ . [ssl at or near Tr, and on previous numerical inves- 
tigations [Til. [T3. [2ll. [23. [5^ . that finite-size corrections 
to the energy behave for all T <Tc according to Eq. ([3]) 
with a; = |. We find this expectation confirmed within 
numerical accuracy, see Fig. |4] (left). Fig. [4] also shows 
that a higher-order correction according to Eq. ^ even 
improve on this leading behavior substantially, which is 
plotted as fit to the data in Fig. [51 
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e„ = e +A N 



0.65 0.66 0.67 0.6S 0.69 0.7 



Figure 4: Various fits to tlie average ground state energies {eo)jv of SK in Tab. IVII Left-most, we consider first-order corrections 
only for a fit involving the thermodynamic limit- value Cao and the correction amplitude A for varying the correction exponent uj. 
The best-quality fit to this form, possessing minimal 'x^ /ndf , occurs just above | at cj ~ 0.68. Adding a higher-order correction 
(with coefficient B, middle panel), a square of the first-order term, bottoms out at oj ~ 0.66 with much lower /ndf values 
overall. Instead, fixing the first-order exponent to cj = | and varying the second-order exponent wi (right panel) implicates an 
optimal exponent much above 2ui, namely uj\ ~ 1.8. Remarkably, the optimal choice for uj or loi about coincides in all three 
fits with the lowest relative error e (fioo) (lower panels) in the fitted value for e^o- 
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Figure 5: Extrapolation plot of the average ground-state 
energy densities (eo)^ from Tab. IVII as a function of the 

presumed finite-size corrections, l/N^. The statistical er- 
rors indicated are much smaller than symbol sizes. For 
TV — > oo, the EO-data extrapolates to the Parisi energy [sij . 
(^o)oo = —0.7631667265(6), at the intercept. The indicated 
fit (dashed line) predicts eoo = —0.76323(5). The slope of the 
line is A = 0.70(1), consistent with the inset, which shows 
the same data appropriately rescaled to extrapolate for A. It 
is B « 0.48, and uji ~ 1.8 suggests surprisingly weak higher- 
order corrections, see also Fig. 2] 



We can obtain a revealing insight into the quality of the 
EO-data by extracting the leading behavior to explore 
the correction term in more detail. Because the energy 
in the thermodynamic limit is well-known (sij . (eo)^ — 
-0.7631667265(6), we can rewrite Eq. ^ as 



A 



[(eo) 



N 



|iV3 



(7V-^oo), (8) 



and plot the EO-data in this form in the inset of Fig. [S] 
Since the form of higher-order corrections are unknown, 
we plot the data again as a function of l/Ns , which pro- 
vides a near-linear collapse of the data and an extrap- 
olated value for the amplitude A « 0.695(5), which is 



consistent with the value obtained by the fit in Fig. [5] 
But the most important aspect of the inset resides in the 
sharp crossover in the behavior of the data at around 
N sa 1000. The consistent behavior of the data for 
N < 1000 suggest sufficient numerical accuracy in the 
obtained ground state energies to this level of analysis, 
without any discernible systematic bias. The data points 
for N — 799 and 1024 both exhibit a systematic error 
of about AA/A ss 2/70 w 3% in the prediction of A, 
hence, a relative systematic error of e (eoo) = Aeoo/eoo ^ 
AA/N^^^/eoc ~ 0.03% (see also Fig.gD in the prediction 
of typical ground state energies overall. Unfortunately, 
the systematic error for the N = 2047 data point is about 
AA/A « 1/7 « 15%, leading to a relative systematic er- 
ror of 0.1% in the prediction of putative ground states, 
which is sufficiently noticeable in Fig. [5] to exclude that 
point from the extrapolation. 

It is worthwhile to compare the inset of Fig. [S] with 
the corresponding plot, the inset of Fig. 1, in Ref. [l^ . 
While the data there is also falling for increasing TV [ar- 
guably to the same asymptotic value oi A ^ 0.7, see 
Eq. ([8])], the variation of the data there is far more rapid. 
Point for point, the data there represents a systematically 
higher value in the average ground-state energy than is 
obtained here. This could potentially indicate a bias in 
the heuristic methods used, which may fail to find true 
ground states across the board. (Notably, the data there 
does not show drastic degradation in the quality of the 
results for increasing N as is found here.) Alternatively, 
such disagreement could be attributed to the difference in 
the bond distribution used: Gaussian there and bimodal 
here. Although the leading thermodynamic properties 
should be universal, higher-order corrections can be sen- 
sitive to microscopic details. 

In Fig. Ul we present an alternative procedure to ex- 
plore corrections that also allows a probe of higher-order 
terms. In this procedure, we select (the most important) 
one of the parameters to be fitted as fixed and evaluate 
the quality of the fit for the remaining parameters over a 



9 



10" 



a> 10" 



10' 



10^ 



., 1 , , , 1 , 1 , 1 , 1 , 

^ SK y 






■— ■ N= 9 


\ 


r 


•— • N= 41 






" N= 63 


w 




^ N= 89 






" N= 127 






^ N= 169 




JsV 


• — • N= 225 






• — ■ N= 299 


\ ■■•\\ " 




" N= 399 


\ - 




— N= 549 


■ 




•-^ N= 649 






►N= 799 




, 1 , 1 , 1 , 1 , 1 


1 , 1 , 1 , 1 , 1 , \ " 



0.84 
0.83 
Q.0.82 
1^ 0.81 
g0.80 
0.79 
D 0.78 
0.77 
0.76 



Bethe r=3 
SK 



l-i 500 200 
N 



Figure 6: Plot of the probability density functions (PDF) of 
obtained ground-state energy densities eo for SK in units of 
the standard deviation a. For reference, the exact probabil- 
ities for A'' = 9 are re- plotted from Ref. [55l |. Unlike for the 
PDFs for GBP in Fig. [T] there is a significant finite-size effect 
noticeable especially in the right tail of the distribution. 
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Figure 7: Double-logarithmic plot of the data from Tab. IVII 
for (jjv (eo) as a function of TV. The data is fitted by a first- 
order fit with a -f hN~f restricted to 100 < A/' < 800 (dotted 
line), giving p k, 0.76 with a /ndf ~ 0.24 for ndf — 13. 
Second-order fits with a + hN"'' + cN~"- allowing all < 800 
and fixed p = | (dash-dotted line) or p = | (dashed line) both 
give essentially indistinguishable results with a /ndf « 0.7 
for ndf — 25 in either case. For p = | we find a ~ 1.3, and 
for p = I it is quite consistent with unity, a ~ 1. 



range of values for the selected one. As a measure of qual- 
ity, we utilize per numbers of degree of freedom (ndf), 
which should be minimized. The first panel displays this 
procedure for just the first-order correction, again con- 
firming the expectation of Eq. ([3]). In the remaining two 
panels we test possible higher-order corrections. In the 
first of these, we test a fit to a regular Taylor series in 
powers of iV~" to second order. Incorporation of such 
a second-order term improves the quality of the fit no- 
ticeably over the first-order term alone. Furthermore, 
the optimal choice for lo again proves consistent with |, 



Figure 8: Extrapolation plot according to Eq. ([6]) towards a 
prediction of p at the ordinate-intercept for SK (black circles) 
and spin glasses on Bethe lattices (blue squares). The abscissa 
denotes the system sizes A'^ on a scale of 1/ log A'^. A linear 
fit (red-dashed line) of the SK data in the apparent scaling 
regime for A > 80 predicts a value of p ~ 0.76. The inclusion 
of the alternative value p = | (dotted line) here shows the 
dramatic change required for the data to attain such a value. 
Yet, a fit (black-dashed line) according to Eq. ([6]) involving 
non-linear corrections down to the smallest A is possible, if 
p = I is assumed. The corresponding situation for Bethe 
lattices makes such an extrapolation further plausible. 



despite the extra degree of freedom provided, which at- 
tests to its robustness. Then, taking a; = | as a given, 
we explore an independent second-order correction with 
scaling exponent wi. This yields the highest-quality fit 
thus far, also used in Fig. [SJ but predicts that wi would 
be much larger than simply 2cj, suggesting that such cor- 
rections would be even weaker. 

We have also tried to fit higher-order corrections of 
the form 1/A^ or \x\N/N in addition to l/N^ correc- 
tions, which are plausible by analogy with the results 
obtained for finite-size corrections near the critical tem- 
perature (5^ [53|. A fit to Eq. ^ does not produce ac- 
ceptable results compared to those found in Fig. U for 
any value of w. Instead, a fit to Eq. ([4]) with wi = 1 
fixed produces a very narrow window of reasonable re- 
sults near a; = | but at best of the quality of what 
is seen correspondingly &t uji — \ in the last panel of 
Fig. U) Further higher-order corrections may improve on 
this alternative. But if the ratio between a previous and 
its next higher-order correction is a weakly falling func- 
tion, i. e. {l/N) / {l/Ni^ = N-^ « 0.1 at least for 

A^ w 1000 here, resolving the impact of such corrections 
with the available data becomes near impossible and they 
can never be fully excluded. 



B. Ground State Energy Fluctuations 

Next, we consider the distribution of ground state ener- 
gies around their averages. In units of their standard de- 
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Figure 9: Data collapse on logarithmic scale of all SK data 
for energies smaller than the average from Fig. [G] On top, 
rescaled by an appropriately chosen Pmax, the data collapses 
onto a power-law curve, which exhibits scaling over more than 
a decade. (Transient data points with P too close to Pmax 
have been removed for clarity.) The collapse is most consis- 
tent (see dash-dotted line) with = | and differs, although 
only slightly, from = | (dashed line). On the bottom, the 
same data (for A'' > 100) is plotted on a linear scale. In this 
form, p — J is clearly more consistent with linear scaling 
(dashed straight lines guide the eye). 
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Figure 10: Data collapse on logarithmic scale of all SK data 
for energies larger than the average from Fig. [S] Rescaled 
by an appropriately chosen Pmax, the data collapses onto a 
power-law curve, which exhibits scaling over more than a 
decade. (Transient data points with P too close to Pmax have 
been removed for clarity.) The collapse is most consistent 
(see dash-dotted line) with — | and differs, again only 
slightly, from = ^ (dashed line). On the right, the same 
data (for A'^ > 50) is plotted on a linear scale. In this form, 
p = I is clearly more consistent with linear scaling (dashed 



straight lines guide the eye) 



viation, the probability density function (PDF) for each 
value of N exhibits clearly the asymmetric shape that is 
skewed towards a broader (exponential) tail of instances 
with lower than average ground state energy and a cut- 
off that is much sharper than exponential for those with 
higher energy. This shape is largely unchanged across the 
sizes and can be shown, by exhaustive enumeration (55| 
of the entire ensemble for A'^ < 9, to arise already for very 
small N. In Fig. [6] we show the PDF for all system sizes, 
which demonstrates the skewness and the small variation 
of the shape with N. Yet, finite size effects larger than 
for GBP in Fig. [Demerge deep in the tails of these PDFs. 

Unlike the overall shape of the PDF for energy fluctu- 
ations, their actual width, measured in terms of the stan- 
dard deviation apf (eg) in Eq. varies in a characteris- 
tic way with N. A plot of the data for ctat (eg) in Tab. I VII 



on a double- logarithmic scale in Fig. [7] suggests a power- 
law decay with N, but the data does not exhibit purely 
linear behavior on this scale, as a simple fit reveals. Only 
when we restrict to iV > 100, a fit of the data accord- 
ing to Eq. ^ with just the leading (p-dependent) term 
provides satisfactory results, with a value of p just above 
I . A more consistent fit of all the data is provided when 
higher-order corrections are considered. This can only 
succeed for a reasonable, fixed value of p, we indeed ac- 
complish almost identical fits of this sort for either p = | 
or yO = I (and probably any nearby value), see Fig. [T] In 
this regard, the fit for fixed p = | has the added benefit 
that the higher-order term appears to scale with a ~ 1, a 
likely candidate for a next-order correction. But if lead- 
ing and next-order correction are that close, for instance 
N~6 /N~^ ~ iVs, to obtain the asymptotic scaling of p 
separated by a decade from any transient behavior would 
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require results for A'' > 10^. Thus, with the present data, 
a conspiracy between such terms leading to the observed 
behavior could not be excluded. 

To illustrate the difficulty more clearly, we follow the 
procedure for GBP in Sec. [5] and extrapolate for p ac- 
cording to Eq. 16]). The variables y = — logcr/logiV 
plotted vs a; = 1/ log — >■ should provide an asymp- 
totically linear extrapolation (with exponentially small 
corrections ~ xe"^""'*'/^, if a > p) towards the expo- 
nent p at the y-intercept for — >■ oo. Plotting the SK 
data up to N K, 1000 in this fashion in Fig. |S] again indi- 
cates a value just above p = | and apparently far below 
p = |. But unlike the GBP data in Fig. [21 the data 
for SK has still transient features even for such large val- 
ues of N . Only a non- linear fit according to all three 
terms in Eq. ([6]), but taking an already fixed p = | as 
given, makes such a high value for p plausible. Further 
support for such a higher value of p is provided by the 
following study of spin glasses on Bethe lattices. On the 
other hand, the GBP example above and the result for 
the TO-vector model of p = | would suggest that an 
altogether different value of p between these two rational 
values is conceivable. In fact, a purely linear extrapola- 
tion in Fig.[5]and the fit in Fig.[7]for N Z 100 would lead 
to an asymptotic value for p very close to that of GBP 
in Sec. IIIII above. 



C. Extreme Fluctuations 

Considering the large amount of data we have obtained 
for SK, we can inspect further details of the energy fluc- 
tuations. In particular, we can look deeper into the tails 
of the PDFs displayed in Fig. IH where they are rescaled 
by their respective crjv. Here, we treat these PDFs un- 
rescaled, according to the form proposed in Rej ^. I24il , sug- 
gested by the spherical spin glass [5a| . In Ref . [2J| , it was 
argued that for ground state energies lower than the av- 
erage, the corresponding branch far in the negative tail 
of each PDF falls exponentially with an argument pro- 
portional in iV, while configurations with larger energies 
are particularly rare for larger system sizes such that the 
positive tail is suppressed by a factor iV^. This system- 
size dependence is largely lost when each PDF is rescaled 
by its width un in Fig. [51 

In Figs. [HI and [TUl we extract the argument of the ex- 
ponential tails and plot the data for each tail reduced by 
the indicated power of N . In the process, the PDF-data 
P(e) for each system size has to be gauged by an arbitrary 
reference point Pmax and transient behavior too close to 
the average or statistically deficient data too deep in the 
tails has to be discarded. The resulting collapse of all the 
intermediate data onto a power-law function is presented 
in the upper panel of each figure on a logarithmic scale. 
According to Ref. [l^l , the power-law exponent can be in- 
terpreted as p~^ for e <C (e) and as p~^ for e ^ (e). To 
provide a reference, we have include lines corresponding 
to p = I and p = I in these plots. While the differ- 



ences are again slim (and it could be argued that true 
asymptotic behavior has not been reached), the consis- 
tent scaling collapse of this vast amount of data seems to 
favor a value closer to p = | again. Furthermore, both 
tails independently exhibit similar scaling behavior. 

When viewed on a linear scale, by taking the respective 
power, only a value closer to p = | provides consistent 
linear behavior for the extant data (see the lower panel of 
Figs.l^and fTU)) . In Ref. [24|, a similar linear plot was pro- 
vided (for the positive branch only) in which the largest 
system size considered there, N = 150, was judged con- 
sistent with p = |. It is clear from our direct compar- 
ison here, that even with the vast amount of additional 
data an ultimately conclusive decision on the true value 
of p is elusive here. The comparison also shows that an 
analysis of these tails on a logarithmic scale is favorable 
over a linear scale which squashes the most interesting 
data points for larger system sizes. Even on a logarith- 
mic scale, though, it is not easy to extract the relevant 
asymptotic information as ever deeper in the tails, only 
ever smaller-sized systems contribute. But overall, in this 
data, small-sized and large-sized systems seem to follow 
similar scaling and project a self-consistent picture. 

V. SPIN GLASSES ON BETHE LATTICES 

To provide a new perspective on the ground-state en- 
ergy fluctuations in SK, we revisit spin glasses with ± J- 
bonds on Bethe lattices (SG), in particular, on those of 
degree r = 3. A similar study has been undertaken in 
Ref. f29|,|3Q|, which concerned thermodynamic averages of 
ground state energies and entropies. Here, we extend the 
sampling of ground state energies to measure the PDF of 
ground state energy fluctuations and the scaling of their 
width. To that end, in Tab. IVIII we have added a large 
number of instances at each system size, up to = 4096. 
Unlike for Bethe lattices of higher degree, at degree three 
we can utilize exact methods [s^ |40, [13| to reduce the 
number of variables in the optimization problem by about 
42%, hence, making larger system sizes accessible at suf- 
ficient statistics. 



A. Average Ground State Energy 

In Fig. 1111 we present the average energy densities ob- 
tained with EO at the system sizes simulated. As in 
Ref. [SP] , the extrapolation of the data is virtually linear 
when plotted as function of N^^ . Such a linear extrapo- 
lation yields (es)^ = —1.2715(1) for the thermodynamic 
energy density, consistent with the value determined 
in Ref. and consistent with the one-step replica- 
symmetry breaking result reported in Refs. [l^[5l|. Re- 
markably, an attempt at adding a higher-order correction 
term contrasts with the same discussion for SK. Nei- 
ther of the two types of higher-order fits presented in 
Fig. m provide reasonable results here. In turn, a fit to 
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Table VII: List of all the data obtained with EO for Bethe 
lattices of degree r — 3 for various system sizes A'^. Given 
are the number of instances ni considered at each A'', and 
the average ground-state energy density (eo) and standard 
deviation cr(eo) over these instances. 



N 


m 


{eo)jv 


cr(eo) 


16 


1000000 


-1.1467(1) 


0.10177 


32 


1000000 


-1.19375(6) 


0.06003 


44 


1680320 


-1.20895(4) 


0.04695 


64 


1000000 


-1.22314(4) 


0.03515 


80 


1486688 


-1.23005(2) 


0.02952 


128 


2000000 


-1.24153(1) 


0.02043 


160 


3273585 


-1.24583(1) 


0.01714 


256 


1498807 


-1.25296(1) 


0.01181 


350 


4015909 


-1.25660(1) 


0.00921 


512 


7638942 


-1.26007(1) 


0.00680 


750 


1718511 


-1.26274(1) 


0.00501 


1024 


743404 


-1.26444(1) 


0.00390 


2048 


113389 


-1.26710(1) 


0.00226 
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Figure 11: Extrapolation plot of the average ground-state 
energy densities (eo)^ for Bethe lattices of degree r = 3 as a 

function of the presumed finite-size corrections, l/Na , As in 
Fig. [5] the statistical errors indicated are much smaller than 
symbol sizes. For A'' — > oo, irrespective of the order of the fit, 
the data extrapolates to (eo)^ = —1.2715(1) at the intercept. 



Eq. which failed for SK, does converge on this data, 
see Fig. [121 Across the plotted regime, the thermody- 
namic value for Coo remains quite robust. The optimum 
is rather close to w = |; such a 2n(i-order fit including lj 
also shown in Fig. [TT] converges to w sa 0.677. 



B. Ground State Energy Fluctuations 

In Fig. I13[ we show the probability density functions 
(PDF) of the energy densities around those averages. 
Overall, those PDFs are a bit more symmetrical than 
for SK in Fig. [51 but exhibit even more finite-size effects 
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Figure 12: Second-order fit over a range of fixed uj, with 
rithmic corrections, to the average ground state energies ( 
for spin glasses on Bethe lattices of degree r = 3 in Tab, 
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Figure 13: Plot of the probability density functions (PDF) of 
obtained ground-state energy densities eo for the spin glass 
on a Bethe lattice of degree r = 3 in units of the standard de- 
viation a. Here, in comparison with Figs.[l]and [6] significant 
finite-size effects are detectable in both tails. 



in both tails, especially in comparison with the corre- 
sponding PDFs of GBP in Fig. [1] Despite their more 
symmetrical appearance, the scaling with N of the de- 
viations a listed in Tab. IVIII seems to indicate an even 
higher value of p, as Fig. [51 suggests. There, those cr for 
the Bethe lattice are displayed in an extrapolation plot 
together with that of SK, to highlight their similarity. 
This data is somewhat smoother than for SK, but just 
as much beset with transients. A family of extrapolants 
for each degree r seems conceivable, reaching all the way 
to the SK-limit at r = oo. In parallel with the discussion 
for SK in Sec. IIVBI we can at best argue that p = | is 
consistent with the trend of the extrapolation. In this 
plot, a value of p = | or even that from GBP seems to 
be ruled out by that trend. 
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-(e-<e>) 

Figure 14: Data collapse on logarithmic scale of all Bethe lat- 
tice data for energies smaller than the average from Fig. 1131 
Top, rescaled by an appropriately chosen Pmax, the data col- 
lapses onto a power-law curve, which exhibits scaling over 
more than a decade. The fit (straight line) gives an exponent 
of ~ 1.36. Bottom, the same data is plotted on a lin- 
ear scale. In this form, p = | is more consistent with linear 
scaling (dashed straight lines guide the eye). 
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Figure 15: Data collapse on a logarithmic scale (top) of all 
Bethe lattice data for energies larger than the average from 
Fig. 1131 Rescaled by an appropriately chosen Pmax, the data 
collapses onto a power-law curve, which exhibits scaling over 
a decades. Bottom, the same data is plotted on a linear scale. 
Here, both set of data are somewhat consistent with linear 
scaling, but the situation is clear than in Fig. [14] (dashed 
straight lines guide the eye). 



C. Extreme Fluctuations 

We have attempted a detailed analysis of the tails of 
the fluctuations for Bethe lattices with the identical ap- 
proach as conducted in Sec. IIV CI for SK. The results 
shown in Figs. [T4lfT5l are indistinguishable from those for 
SK above, and would also suggest a value closer to p = | 
for the Bethe lattice, which seems to contradict the indi- 
cation provided by the extrapolation of a in Fig. [S) 



VI. SPIN GLASSES ON RANDOM GRAPHS 

As a useful reference point to the previous studies, we 
also include a comparison with a spin glass on sparse, 
ordinary random graphs of mean degree C = 2. It pro- 
vides an example where fluctuations in the ground states, 
whether the energy or the cost, appear to converge to a 
normal distribution. As argued in Sec. Ill Bl it is essen- 



tial for the case of a fluctuating geometry to focus on the 
actual cost, i. e. the total absolute weight of the violated 
bonds, of the ground state. Still, even the PDF for these 
ground state costs seems asymptotically Gaussian, as has 
been predicted recently in Ref. [25|. 



A. Average Ground State Costs 

In Tab. IVIIII we have listed the average ground state 
costs and their deviations for a number of system sizes 
up to TV = 4096. A large number of instances has been 
averaged over, even at the largest sizes, since the exact 
graph reduction methods [s^ ll^, [l^l used for the Bethe 
lattice above are even more effective here: Even at the 
largest size, those reductions result in graphs of at most 
15% of the original size that need to be optimized with 
the EO heuristic. 

In Fig. [m we plot the extrapolation of those average 
ground state costs to the thermodynamic limit. Again, 



14 



Table VIII: List of all the data obtained with EO for spin 
glasses on random graphs of average degree C = 2 at system 
sizes A*'. Given are the number of instances ni considered at 
each A'^, and the average ground-state cost density (co)jy smd 
average standard deviation a{co) over these instances. 



iV 


m 


{eo)]v 


crjv (eo) 


64 


1050000 


0.05504(2) 


0.01693 


128 


1050000 


0.04994(1) 


0.01062 


180 


1050000 


0.04808(1) 


0.00846 


256 


1050000 


0.04652(1) 


0.00671 


360 


1050000 


0.045296(5) 


0.00537 


512 


1050000 


0.044266(4) 


0.00429 


1024 


395722 


0.042808(4) 


0.00278 


2048 


668638 


0.041883(4) 


0.00183 


4096 


319036 


0.041304(4) 


0.00123 
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Figure 16: Extrapolation plot of the average ground-state cost 
densities (co)jv for ordinary random graphs of average degree 
C = 2 as a function of the presumed finite-size corrections, 
1/N3. As in Fig. [5] the statistical errors indicated are much 
smaller than symbol sizes. Shown are also a first-order (blue 
dash-dotted line) and a second-order fit (black dashed line) 

2 

in powers of l/Ns. For N ^ oo, the data extrapolates to 
(co)^ = 0.04030(5) at the intercept. 
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Figure 17: Second-order fit in powers of 7V~" over a range 
of fixed iu, to the average ground state costs {co)j^ for spin 
glasses on random graphs of degree 2 in Tab. IVIIII The min- 
imum in x^/ndf in the upper plot strongly suggests a pure 
power series with tj = |; such a fit is included in Fig. 1161 
The lower panel shows the range of extrapolated values in 
the thermodynamic cost density. 
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Figure 18: Plot of the probability density functions (PDF) of 
obtained ground-state cost densities co for the spin glass on 
an ordinary random graph of average degree 2 in units of the 
standard deviation a. 



the extrapolation proves most consistent with N^s cor- 
rections at finite size, although stronger transients are 
apparent here. In fact, Fig. [17] indicates that finite-size 
corrections may be a pure power series in iV^a , the first 
two orders of which are also shown as asymptotic fits in 
Fig.[lll 



B. Ground State Cost Fluctuations 

In Fig. [THl we have plotted the PDF for the ground 
state cost fluctuations. It shows no sign of asymmetry for 
any size N. Therefore, it is quite surprising that the finite 
size values obtained for the deviation (cq) plotted in 
Fig- US and extrapolated in Fig. [50] exhibit a rather slow 



convergence to a normal width. These results serve as a 
warning how, even for seemingly trivial fluctuations, the 
asymptotic behavior miglrt be reached only quite slowly. 



VII. THREE-SPIN INTERACTIONS ON BETHE 
LATTICES 

To complement the discussion of glasses with two-spin 
interactions, and to compare with related work ^31i. i5^. 
we have also considered spin glasses on Bethe lattices 
with three-spin interactions, both, with discrete (±J) 
and Gaussian bonds, as listed in Tabs. IIXI and [X] This 
study succeeds in confirming recent observations regard- 
ing finite-size corrections to the average ground state en- 
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Figure 19: Double-logarithmic plot with system size A*' of the 
deviations in the ground state costs, here given as N^om (cq) 
to highlight any difference from Gaussian behavior, for a spin 
glass on an ordinary random graphs of average degree C = 2. 
The plot is far from flat or linear, suggesting significant cor- 
rections to scaling. Expecting asymptotically normal scaling 
with p = |, we fitted the data with an additional higher-order 
correction term with ~ A*'"" (dashed line), as in Eq. ([2]). The 
fit determines a ~ 0.75. 
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Figure 20: Extrapolation plot of the data for the deviation a 
in Tab. I Villi according to Eq. A fit is only possible for a 
fixed p = \- 



ergy [sij. But they can merely give a crude picture of 
deviations within the distribution of those energies. This 
inadequacy has two origins: First, it proves inherently 
challenging to determine ground states for instances of 
this problem with any accuracy already at moderately 
sized systems. The systems are large enough to predict 
average energies with reasonable errors, but insufficient 
for the asymptotic analysis for the deviations. Second, 
those deviations in their own right appear to be far nar- 
rower for this problem than for any of the two-spin mod- 
els above. In fact, for discrete bonds the grounds states 
seem to cover only a few states above and below the av- 
erage, with almost an invariant width, such that the de- 



Table IX: List of all the data obtained with EO for p = 3- 
spin glasses on Bethe lattices of degree r = 4 at system sizes 
TV for discrete bonds. Given are the number of instances nj 
considered at each A'^, and the average ground-state energy 
density {eo}j^ and standard deviation a over these instances. 
The largest system size has unacceptable systematic errors 
and is ignored in any fit. 



iV 


m 


(eo)iv 


o-jv (eo) 


iV 


71/ 


{eo)]v 


o-N (eo) 


15 


10000 


-1.104(1) 


0.0995 


51 


130000 


-1.1760(1) 


0.0296 


18 


100000 


-1.1171(3) 


0.0834 


54 


800000 


-1.17843(3) 


0.0267 


24 


100000 


-1.1406(2) 


0.0584 


60 


25000 


-1.1829(2) 


0.0242 


30 


30000 


-1.154(3) 


0.0500 


75 


400000 


-1.18892(3) 


0.0189 


33 


100000 


-1.1576(2) 


0.0460 


90 


35000 


-1.1929(1) 


0.0157 


36 


100000 


-1.1619(1) 


0.0411 


120 


11000 


-1.1973(1) 


0.0121 


39 


100000 


-1.1665(1) 


0.0365 


150 


200000 


-1.20087(2) 


0.0103 


45 


30000 


-1.173(2) 


0.0329 


180 


80000 


-1.20264(3) 


0.0087 


48 


800000 


-1.17435(3) 


0.0319 


240 


350 


-1.1995(4) 


0.0067 



Table X: List of all the data obtained with EO for p = 3- 
spin glasses on Bethe lattices of degree r = 4 at system sizes 
TV for Gaussian bonds. Given are the number of instances 
ni considered at each TV, and the average ground-state cost 
density (co)jv and standard deviation a over these instances. 
The largest system size has unacceptable systematic errors 
and is ignored in any fit. 



TV 


71/ 


{co)n 


o-N (co) 


33 


100000 


0.02366(3) 


0.00916 


39 


120000 


0.02261(2) 


0.00783 


51 


120000 


0.02122(2) 


0.00616 


66 


80000 


0.02016(3) 


0.00495 


99 


25000 


0.01901(2) 


0.00356 


144 


3500 


0.0193(1) 


0.00274 



viations in the density seem to fall with ^ 1/TV. 

A somewhat wider distribution is observed for Gaus- 
sian bonds, which provides for a smoother appearance 
for the PDF at all system sizes compared to the dis- 
crete case. Any skewiiess can only be observed when 
plotted for ground state cost fluctuations; energy fluc- 
tuations would always be normal, originating from the 
random fluctuations in the total bond-weight themselves, 
as described in Sec. Ill Bl Surprisingly, these cost fluctu- 
ations skew exactly in the opposite direction from any 
previous studied PDF, such as those above. Although 
the deviations a extracted from those PDFs indeed seem 
inconsistent with 1/TV scaling, the system sizes attained 
in this study are rather small, TV < 100, and asymptotic 
behavior may not have been reached in this study. 



A. Ground State Energy 

In Figs. I^ni^ we extrapolate the obtained average 
ground state energies (for discrete bonds) or costs (for 



16 



-1.14 

-1.15 

-1.16^ 

-1.17 

a.*-1.18 
V 

-1.19r 
-1.20 r 
-1.21 
-1.22 



EO 
fit 



p=3, r=4 (±J) 







0.01 



0.02 

1/N 



0.03 



0.04 



0.10- 



0.01 




Figure 21: Extrapolation plot of the average ground-state 
energy densities (eo)jv for a spin glass with discrete bonds 
in which p = 3 spins mutually interact on a Bethe lattices 
of degree r = 4, as a function of the presumed finite-size 
corrections, 1/A'^. 



Figure 23: Log-log-plot of the deviations a in the ground state 
energy fluctuations of a p = 3 spin glass on Bethe lattices of 
degree r = 4 as a function of system size A'^. The data is very 
difficult to fit, and we only provide the dashed line ~ 1/A'^ as 
a guide to the eye. 
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Figure 22: Extrapolation plot of the average ground-state 
cost densities (eo)^ for a Gaussian spin glass with three-spin 
interactions on a Bethe lattices of degree r = 4, as a function 
of the presumed finite-size corrections, 1/A''. 



Gaussian bonds) in Tabs . HXl and 1X1 In both cases, finite 
size corrections appear to decay with volume corrections, 
hke 1/A^, stronger than in any of the p ~ 2-spin models 
above. Hence, even though the system sizes are small, 
quite reasonable extrapolations are achieved. In the dis- 
crete case in Fig. there appears to be significant struc- 
ture in the transients. By reproducing the exact ground 
states for a sample of those instances up to size = 51 
with exact (branch-and-bound) algorithms, we have ver- 
ified that these are not due to systematic errors in the 
optimization heuristic, as 100% agreement was achieved 
for each instance. Rather, we expect that those effects are 
due to the constraints in the formation of 4-regular hyper- 
graphs at small N in combination with the discrete set 
of bonds available. Correspondingly, the Gaussian data 
is free of any such structure, and thus extrapolates with 



Figure 24: Log-log-plot of the deviations a in the ground state 
energy fluctuations for a.p = 3-spin glass on Bethe lattices of 
degree r = 4 as a function of system size A'', here for Gaussian 
bonds. As for the plot for discrete bonds in Fig. 1231 this data 
is also difficult to fit, and there is a visible trend to even lower 
values in the exponent then 0.87 found by only fitting to sizes 
iV < 100 (dashed line). 



comparable accuracy despite the overall smaller system 
sizes used. 

For N — oo, the discrete data in Fig. [5T] extrapolates 
to (eo)o^ = —1.213(2) at the intercept, slightly above 
the one-step replica symmetry-breaking (RSB) predic- 
tion [5^, as would be expected for the true ground state 
at full RSB. With bonds drawn from a Gaussian dis- 
tribution, we instead plot the actual cost of violated 
bonds. It is difficult to obtain good minima already at 
N K, 100, such that we can only extrapolate data for 
smaller TV. For iV ^ 00, the extrapolation yields a cost 
of (co)oQ — 0.0167(4) at the intercept. Since average cost 
and energy density for Gaussian bonds are related via 
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Figure 25: Plot of the probability density functions (PDF) 
of obtained ground-state energy densities eo, in units of the 
standard deviation (t, for the -p — 3-spin glass with discrete 
bonds on an r = 4-regular random graph. The rugged appear- 
ance of the data is due to the discreteness of the bonds and 
the fact that only a very narrow set of energy values around 
the mean are taken on. 
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Figure 26: Plot of the probability density functions (PDF) of 
obtained ground-state cost densities co, in units of the stan- 
dard deviation a, for thep = 3 spin glass with Gaussian bonds 
on an r = 4-regular random graph. The data is skewed with a 
sharp cut-off for costs less than the average and a long tail for 
larger cost, exactly opposite to any previous PDF for energy 
fluctuations. The inset shows the same data plotted as energy 
fluctuation PDF, which are purely normal distributed. 



Eq. (O, we obtain with (|J|) = r = 4, and p = 3 

that (eo) = -1.030(1). 

B. Ground State Fluctuations 

In light of the limited ability to produce ground states 
at larger system sizes, any prediction for the fluctuation 
exponent p is poor. Furthermore, extreme fluctuations 
are difficult to attain, since the width is rather narrow. 
Yet, for discrete bonds the data plotted for a in Fig. [23] 
is quite consistent with 1/iV-decay, i. e. p = 1. But there 
does not seem to be a clear trend towards asymptotic 
scaling in the case of Gaussian bonds shown in Fig. [531 If 
anything, the data appears to exhibit upward curvature, 
away from an 1/iV scaling regime, indicating that p in 
this case may be even lower than the fitted value of 
0.87. Such a discrepancy between discrete and Gaussian 
bonds on Bethe lattices was also noted for the p — 2-spin 
model m. 

With the rapid decay of the width for discrete bonds, 
it is not surprising that the PDF for its energy fluctua- 
tions has a somewhat rugged appearance: only few, dis- 
crete energy values can be taken on left or right of the 
mean. As shown in Fig. [53 the PDF otherwise skews 
similarly to all previous cases. It comes as a surprise 
then, that the corresponding PDF for Gaussian bonds in 
Fig. [551 skews exactly in the opposite direction. Although 
it is plotted for the cost fluctuations instead - the energy 
fluctuations in the inset are purely normal -, this does 
not explain the difference in the skewness, since cost and 
energy are linearly related [as in Eq. ([71)]. Lower cost 
correlates with lower energy and vice versa. We specu- 
late that the skewness here has a rather trivial origin: As 



indicated by the very low average ground state cost per 
spin, (co)qq = 0.0167(4) (about 1/4 of that for discrete 
bonds) , below-average-cost instances may be hard to flnd 
due to the proximity of entirely cost-free, "perfect" solu- 
tions (although we have not actually generated a single 
solvable instance during our study). On the other hand, 
higher-cost instances are likely produced by the addition 
of isolated "defects" in the quenched bonds that can not 
be gauge-transformed away as easily as in SK ^] (or 
probably other problems), where many such frustrated 
plaquettes overlap. The smooth and well-converged ap- 
pearance of the PDF in Fig. 1261 may provide some confi- 
dence that the lack of scaling in the respective deviations 
a in Fig. [531 is not due to a systematic error in the sam- 
pling. It is the nature of heuristic optimization that any 
bias at large system sizes (larger than can be verified by 
exact methods) can never be exclude. But as the exam- 
ples in the previous sections demonstrate, strong com- 
petition between correction terms may be equally well a 
cause in precluding asymptotic scaling. 

We can address the puzzling skewness for the Gaus- 
sian case further by comparing with previous results with 
Gaussian bonds on p = 2-spin glasses on Bethe lattices. 
While we have experienced this case already for discrete 
bonds at r = 3 above in Sec. [Vl without any qualita- 
tive difference in behavior compared to the SK model, its 
Gaussian version presents a very odd pattern. In Ref. (32| 
we have already remarked on the unusual finite-size cor- 
rections to the thermodynamic average ground state cost 
or energy and the confusing trend that already beset the 
cost deviations a (cq). We therefore missed the even more 
surprising evolution with r and N of the skewness in the 
corresponding PDFs. Aside from some finite-size effects, 
all PDFs for the energies are approximately normal dis- 
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Figure 27: Plot of the PDFs for the fluctuations of the ground state costs per spin for a p — 2-spin glass with a Gaussian bond 
distribution on Bethe lattices of degree r = 3, 4, . . . , 10, 15. For smaller r and sizes A'^, the PDFs are right-skewed, just as in 
Fig. 1261 but especially for larger r those PDFs appear to approach a normal distribution for large sizes A'^. 



tributions. But the PDFs for cost fluctuations shown in 
Fig. 1271 range from those strongly skewed, comparable to 
Fig. [551 for small r to those with only mild skewness at 
larger r. For all cases, but most drastically for larger 
r, increasing system size N symmetrizes the PDFs to- 
wards an apparent normal shape. Note that for the SK 
limit r — > oo, these distribution should approach a nor- 
mal form for the cost. While energy fluctuations should 
become the non-trivial PDF in the SK limit, r = 15 is ap- 
parently not large enough for this effect to be discernible. 



VIII. CONCLUSIONS 

To elucidate the origin of the unusual ground state de- 
viations observed in the SK model, we have parsed over 
a range of different spin glass and combinatorial models 
in an effort to provide context for any typical or atypi- 
cal properties. The clearest picture emerges from graph- 
bipartitioning (GBP) on Bethe lattices, corresponding to 
a ferromagnetic system held at zero magnetization. The 



scaling exponent p for each degree r of the network pos- 
sesses a convincing extrapolation, but to values unlike 
those proposed for SK |20l - l25l |. suggesting a very differ- 
ent origin for the fluctuations in GBP. At best, they come 
closest to the value of p = | recently found for the m- 
vector spin glass ■ The situation for the SK is less 
clear, apparently due to very strong higher-order correc- 
tions in the finite-size behavior. Although far from a 
scaling with the theoretically favored value of such a be- 
havior for the SK becomes plausible with very close 1 /N 
corrections, a scenario further supported by the Bethe 
lattice spin glass at degree r = 3 (with discrete bonds) 
exhibiting very similar behavior but inherently closer to 
p = |. As the example of the vector model demon- 
strates, entirely distinct values are conceivable for any of 
these models, but since in the limit r — >■ oo the Bethe 
lattices approaches SK, such a distinction seems implau- 
sible. A look deep into the tails for SK and the Bethe 
lattices, for which extensive results have beeir generated, 
would suggest that the scaling of the deviations a with 
the exponent p that is dominated by near-typical fluctu- 
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ations may be disconnected from the extremely atypical 
fluctuations deep in the tails, which is more consistent 
with |-scaling for both models. But despite the mas- 
sive amount of data obtained here, any true asymptotic 
scaling for the tails may still be elusive. 

The situation might be simpler for models in which 
more than two spins interact. For instance, we find for a 
p = 3-spin glass model with discrete bonds on a r = 4- 
regular Bethe lattices that fluctuations scale about with 
1/7V, i. e. p = 1, which would already attain the re- 
sult for the REM (p oo) [H in Ref- On the other 
hand, the same model with Gaussian bonds demonstrates 
the fragility of the phenomenon: energy-fluctuations get 
overwhelmed by trivial (normal) fluctuations in the con- 
tinuous bond weights, while the more pertinent cost- 
fluctuations are skewed exactly in the opposite direction 
from those from SK, or even those on the same graph 
with discrete bonds. A same effect is found for p = 2 on 
Bethe lattices at low r. In summary, the large variation 



in behaviors of fluctuations not only between models, but 
even within models for different bond distributions, hints 
at the strong dependence on minute details of the under- 
lying graph geometry and variability in bond weights. 
Either can impact whether and how frustrated plaque- 
ttes correlate to ease the cost when an instance possesses 
more or less of those than the average. 
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